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ABSTRACT 

Learner Verification and Revision (LVR) has been part 
of Florida's instructional materials adoption process since 1974, 
when state statutes required textbook publishers to provide written 
proof that their materials were revised based on student feedback 
before publication. Publishers were also required to collect student 
data and make revisions during the entire market life of the product. 
In 1983, the LVR guidelines were made more specific and required that 
the LVR be submitted by publishers prior to submitting textbooks for 
adoption. The current 1985 Florida LVR guidelines required four 
sections: (1) product background information; (2) intended learner 
outcomes; (3) prepublication LVR— overview of LVR process, data 
collection and use, instruments used, and sample revisions; and (4) 
post publication LVR — plans for further data collection. Training was 
provided for publishers, and state instructional materials councils 
felt it was easier to evaluate the reports. Publishers, however, 
expressed concerns about the cost of field testing an entire 
textbook, validity and reliability of students' feedback, paperwork, 
and lack of evidence validating LVR. In light of these problems, it 
was recommended that the guidelines be reviewed, that publishers work 
with authors in implementing LVR, and that guidelines be applied more 
consistently. (GDC) 
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If there is one state that is serious about improving education, it's 
Florida. For the past three years the Florida Legislature has passed 
initiatives which would put into place a system that would provide required 
student performance standards, quality instructional materials to help teach 
to these standards, and an assessment program to see how well students are 
learning. Standards or objectives, instructional materials, and assessments - 
these are the three basic elements for any good instruction. So Florida seems 
to be on the right track. 

The focus of this paper is Florida's initiatives for improving textbooks. 
Florida's special interest in teaching materials goes back a number of years. 
For over 50 years, Florida has used a state adoption process. A rather 
intricate system that involves a state level council of 9 members, who are 
mostly teachers, decide which books are to be recommended for the state 
adopted list. Input from school districts around the state is used in making 
these decisions. The system works fairly well and school districts get to 
purchase with state funds from a list which may have up to 15 titles for each 
subject area. 

Part of the adoption process is Learner Verification and Revision which 
is familiarly known as LVR. The intent of LVR is to improve the quality of 
textbooks by involving the learner in the development process. LVR has been 
required of publishers since 1974. 

1. Paper presented at the American Educational Research Association 
convention in San Franciso, April, 1986. 

U.l. DEPARTMENT OF EDUCATION 
Office ot Educational Reaetrch and Improvement 

-1— EDUCATIONAL RESOURCES INFORMATION 

* CENTER (ERIC) 

IFThia document hat been reproduced at 
' received from the peraon or organization 
originating it 

2 0 Minor chengea have been made to improve 
reproduction quelity. 

• Pointt of view or opinion t tteted in thia docu* 
men! do not necetaarily rep re tent official 
OERI position or policy. 



The purpose of this paper is to describe and analyze how LVR is used in 
the Florida adoption process* More specifically, tlve paper will (a) define 
LVR and its research basis; (b) describe its roots in Florida Statutes; (c) 
describe the current Florida requirements; (d) problems associated with the 
implementation of the process; (e) and some reconmenda tions . 

Historical Background of the Textbook Revision Process 
Instructors have been changing their approach to teaching based on 
feedback from students for as long as there has been teaching. So the 
selection of a starting point for a description of the historical development 
of the revision process is relatively arbitrary. The current legislative 
interest in the process has its roots in several developments from the 1960s. 
This was a time when the federal government put forth its most significant 
funding of elementary and secondary school curriculum development projects. 
The textbook development model most typically used during that era was to 
bring together the best subject matter experts to write the instruction. The 
materials were edited and prepared for distribution by a project staff or by a 
major publisher that had temporary rights to them. It was soon recognized 
that many of the new texts were effective only with the brightest students. 
Evidence indicated that they were decidedly ineffective with average and below 
average students. 

The issue of the instructional effectiveness of Instructional Materials 
came to prominence in 1967 with the publication of Scriven's paper "The 
Methodology of Evaluation". 1 ' He proposed that the concept of evaluation be 
expanded, as suggested earlier by Cronback, to include the collection of data 
and information to improve instruction. 2 He referred to this process as 
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formative evaluation. Scriven was not proposing that this data be used as the 
basis for making decisions about the current value or worth of the 
instruction. This type of evaluation he referred to as summative. Instead, 
the purpose of formative evaluation, was to provide a stimulus to writers to 
make informed revisions to make the instruction more effective. He argued 
that had this process been used in the major curriculum development projects, 
project directors would have been aware of the limitations of the instruction 
they were preparing. They could then have made significant changes prior to 
the publication of the textbooks and supporting materials. 

While Scriven provided the conceptual rationale for formative evaluation 
and revision, he did not list detailed guidelines for how it was to be done. 
There was another set of instructional development activities in the 1960s 
which would provide this guidance. This was the Skinnerian approach to 
developing programmed instruction. Those who followed this small step, linear 
approach to instruction developed procedures for collecting data on posttest 
performance, answers to attitude questions, and answers written to questions 
which were asked during the learning process. This data was used to pinpoint 
problems in the programmed instruction text and to indicate the type of 
revisions to be made. This developmental testing strategy, as it was called 
by Markle, closely resembles the formative evaluation strategy used today by 
instructional designers. 3 

Thus, by 1967 both the concept of formative evaluation and the process 
for doing it were in place. What brought them together was an emerging 
technology called the systems approach. The most basic systems approach model 
indicated that the developer had to start with a set of objectives for which 
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instruction was to be developed. Following use of the instruction, students 
were tested with items that assessed how well they had mastered the 
objectives, and the instruction was revised in those areas in which students 
did not achieve mastery. The systems approach model requires a testing and 
revision process, or it will not operate like a system. That testing and 
revision process is now commonly accepted to be the formative evaluation 
process described by Scriven and Markle in the 1960s. 

It is important to emphasize that the basic approach in this model is one 
that depends upon input from learners as the crucial component of the 
formative evaluation and revision process. Student incorrect answers to test 
questions and their comments about the materials are the basic elements in the 
revision process. This does not rule out input from subject matter experts or 
teachers, but rather it puts a premium on the responses of learners since they 
are the ultimate beneficiaries of effective instruction. 

The focus on student data was evident in the research that was conducted 
in the 1960s and 1970s. Many studies were done on the general effectiveness 
of programmed instruction. One of the most quoted studies of that era was 
conducted in 1965 by Roebeck.^ This classic study focused on the revision 
process which was then being tried out. Roebeck wanted to determine the 
effectiveness of a revision based upon input from only one student. He asked 
a sixth grade student to complete a prototype copy of a programmed instruction 
text. Based upon the student's feedback, the instruction was revised and 
tried out with a second learner. The feedback was also used to create a 
second revised version of the instruction. 

Hie prototype and the two revisions were then provided to three groups of 
matched students. Analysis of posttest performance indicated that both 
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revised texts resulted in significantly better posttest performance than the 
orignal prototype. There was no significant difference in the effectiveness 
of the two revised texts. This relatively straightforward study is noted here 
because it, and others like it, would be referenced repeatedly in the 1970s as 
evidence of the effectiveness of the formative evaluation process. It was to 
be argued that a revision based upon the input of only one student would 
result in significantly improved instruction. 
MMHCPQLOGy 

The methodology for revising instructional materials is not cut and dry. 
Research has not come up with any simple rules. However, there are generic 
steps that are followed by most instructional designers. The current 
procedures have been greatly influenced by programmed instruction 
practitioners, such as Markle, as well as by the writings of Eva Baker and 
Marvin Alkin. 5 ' 6 The process described here is similar to that described by 
Dick and Carey 7 ' 8 in the second edition of their text on instructional design. 

The emphasis in formative evaluation methodologi s has been on 
prepublication improvement of the instructional package. This takes place 
through the use of evaluation procedures at three stages in the development 
process. The three stages, which occur after an early draft or prototype of 
the instruction has been developed, are referred to as the one-to-one stage, 
the small group stage, and the fi i trial. Each has a special purpose which 
is reflected in the procedures u . 

One-to-one-stage. The initial step in the typical formative evaluation 
process is to use a draft version of the instruction with several learners who 
are representative of the students for* whom the text is intended. The 
learners are carefully selected to represent high, average, and low-ability 
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learners. The developer works with the students individually as they go 
through the materials. 

The students are told that the materials are under development and that 
their help is needed in finding out what is wrong with them. Rapport is 
developed with the learners before the process is started in order to reassure 
them that their input is really wanted. 

Hie developer and the learner sit side-by-side as the student begins to 
go through the instruction. If appropriate, the student may be asked to read 
aloud for awhile to provide a sense of the readability of the instruction. 
The developer may stop the student at prespecif ied locations and ask questions 
about the information that has just been presented. If the student has not 
understood a concept, the developer will often try to restate it in more 
understa n da b le terms. This process is continued until the learner completes 
30 to 90 minutes of instruction. Then a posttest covering the objectives 
taught in the instruction is administered to the student. Each item is read, 
answered, and descussed. Following the test, the developer has a general 
discussion with the learner about the materials. 

The major purpose of the one-to-one stage is to identify the significant 
problems with the instruction and to try out alternative instructional 
strategies on-the-spot. The problems identified may be as trivial as 
typographical errors or missing illustrations, and as important as missing 
pieces of instruction or activities that just don't work. This may be the 
first encounter the developer has with the actual learners, and thus tbeir 
various interests and characteristics may be noted for the first time. 

It should be pointed out that it is the developer who typically conducts 
the one-to-one evaluation, rather than an evaluator who may not be familiar 
enough with the content to interact effectively with the learners. 
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Snail group stage. The second stage in the process is referred to as the 
snail group stage. After revisions have been made on the basis of one-to-one 
evaluations, the instruction is reproduced for use with a large number of 
learners, usually 8 to 20. These numbers are arbitrary, but an attempt is 
made to get a large enough sample so that a sense of group rather individual 
performance can be obtained* 

The term small group does not mean that all the learners must take the 
instruction at one tine, but they may. The term simply refers to the general 
number of participants involved at this stage. The major purpose of this 
second stage is to determine the effectives 3s of the revision made following 
the one-to-one stage, and to see how the instruction works when learners 
proceed on their own. 

Learners who participate at this stage are told the purpose of their 
participation and are asked to go through the instruction whi 1 a the developer 
observes the group. If significant problems arise, the developer can step in, 
but otherwise the learners proceed through it on their cwn. 

After finishing the instruction, they take the posttest and complete an 
attitude questionnaire which probes their reactions to the instruction they 
have just completed. They may also have a debriefing session with the 
developer to discuss the instruction. 

This second stage gives the developer the first real indication of the 
effectiveness of the ins true tior as noted by student posttest performance. It 
also is the first indication -o how well the student can get through the 
instruction without an instrur at their side. The data from this stage is 
extremely valuable for identi 3 the remaining problems with the instruction 
both in terms of the content d the directions to the learners. It is not 



-7- 

8 



unusual to discover that there are still some problems with the tests. 
Revisions which take into account test performance, student attitudes, and 
developer observations are made following the small group stage. 
Field trial stage. 

The third and final stage of formative evaluation is usually referred to 
as a field trial. There are two purposes for this stage, namely, to determine 
the effectiveness of the changes made as a result of the small group 
evaluation, and to determine if the instruction can be used by learners and 
teachers in the normal instructional setting. 

As is implied by the purpose for this stage; the developer has almost no 
role to play other than perhaps as an unobtrusive observer. The instruction 
is tried out by one or more regular teachers with typical students in normal 
classrooms or in other appropriate instructional settings. It is not unusual 
for the teachers to be provided inservice training on the use of the 
instruction prior to the field trial. 

During the field trial the teachers and students use the materials in the 
manner intended by the developer. Note is taken of any problems which may 
affect the usefulness of the materials. As before, the students are tested at 
the conclusion of the instruction, and are ask^ - their views about the 
instruction through an attitude questionnaire or a discussion with the 
teacher. 

The data from one or more sites (seldom are more than five or six 
classrooms necessary) is summarized and given to the developer for use in the 
final round of revisions prior to the first official publication of the 
instructional materials. 



There are a number of important features of the total formative 
evaluation process that should be noted. First and foremost, the entire 
process is driven by data from learners. Student data is the primary source 
of information about what works and what doesn't. While subject matter 
experts must certify to the accuracy and currentness of the instruction, it is 
the students from whom the key data for additional revisions must be obtained. 

It also should be noted that the developer (which in fact, may be a team 
rather than a single individual) is heavily involved in much of the process. 
The lack of objectivity of the developer in the evaluation setting is offset 
by the value of the input received and the ability to immediately translate it 
into revision strategies. 

Hie final point to be made is that throughout the process, the summative 
question about hew good are these materials is never asked. The emphasis is 
continually on the question of how can the instructional materials be made 
more effective. The question of "Are they any good?", must be delayed. And, 
as instructional designers know, summative evaluation data presented by the 
developer of the materials is likely to be suspect anyway. Typically, someone 
else must do the summative evaluation. 
Legislative Initiatives 

The enthusiasm for an empirical process of curriculum development in the 
form of a systems approach not only reached developers and researchers, but 
legislators as well. They were convinced that the formative evaluation 
process, which would take on a new name, should become a required part of the 
oommerical textbook development process. The impetus for this enthusiasm can 
be largely attributed to Kenneth Komoski, Executive Director of Educational 
Products Information Exchange (EPIE). 
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In research conducted by Komoski from 1967 to 1971, he learned that of 
the approximate 200,000 then catalogued commercially produced instructional 
materials on the market, less than 1 percent had ever been tried out with 
learners prior to publication 8 '^. The obvious conclusion was that commercial 
publishers were disregarding the use of formative evaluation procedures in 
their development process. To distinguish the process recommended for 
commerically produced materials from that of the traditional formative 
evaluation process, Komoski coined the phrase "learner verfication and 
revision 11 (LVR). Not only was data to be collected from learners prior to 
publication, but the process was to be ongoing. During the entire market life 
of the product efforts were to be made to continually revise the materials 
through learner feedback. 

Although most researchers agreed with Komoski 1 s basic concept, some had 
difficulty with his terminology. They found it confusing and even misleading. 
Some authors accused Komoski of throwing out sunmative evaluation, replacing 
it with a kind of perpetual formative evaluation. 

Komoski attracted the attention of several state legislatures. 
California enacted legislation in 1972 requiring publishers to provide 
evidence of "learner verification" prior to their books being considered for 
use in elementary classrooms. Unfortunatly, the requirement of publishers was 
interpreted to mean that publishers had to provide proof that the materials 
worked with intended learners. This amounted to a validation of the products. 
As a result, California legislation went fairly well unnoticed and has had 
little or no impact on how publishers collect data to improve textbooks. 
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Florida fared somewhat better. In 1974, with input from Komoski, the 
legislature passed Chapter 233.25, Florida Statutes: 

"Publishers and Manufacturers of instructional materials, or their 
representatives, shall: (3) Submit, at a time designated in Section 
233.14, Florida Statutes, the following information: (b) Written proof of 
the use of the learner verification and revision process during 
prepublication development and postpublication revision of the material 
in question. For purposes of this section, "learner verification" is 
defined as the empirical process of data gathering and analysis by which 
a publisher of curriculum materials has improved the instructional 
effectiveness of that product before it reaches the market and then 
continues to gather data from learners in order to improve the guality 
and reliability of that material during its full market life. Failing 
such proof, if the publisher wishes to submit material for adoption, he 
must satisfy the state instructional materials selection council that he 
will systematically gather and utilize learner verification data to 
revise the materials in question to better meet the needs of learners 
throughout the state 10 . Such text revision should be interpeted as including 
specific revision of the materials themselves, revision of the teachers 1 
materials, and revision of teachers 1 skills through retraining, it being the 
intent of the legislature, that learner verification and revision data shall 
include data gathered directly from learners; may include the results of 
criterion referenced and group normed tests, direct learner comments, or 
information gathered frcm written questionnaires; and may not preclude the use 
of secondary data gathered from teachers, supervisors, parents, and all 
appropriate participants and observers of the teaching learning process". 
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Florida's requirements were much more specific than those of California. 
An effort was made to define the process and to list the types of data 
expected. Also, the law required written proof of the process rather than a 
simple statement that the process had been used. 
LVR Guidelines 

But due to confusion as to the meaning of the LVR process, lack of 
commitment by state department administrators to the LVR process, as well as 
opposition to the basic concept by publishers, the Impact of LVR legislation 
was minor. The guidelines used by the state lacked specificity and the law 
had an escape clause which allowed publishers to submit a post publication LVR 
plan if they had not done LVR on the product. Some publishers took advantage 
of this clause and submitted one planned LVR after another. 

This was the case in Florida until 1983. At that time it was decided to 
strengthen and enforce the LVR legislation. To acccomplish this, the 
guidelines for publishers' reports were made more specific and enforceable. 
The plan approach clause was take out of the law in 1984. Now publishers must 
have completed an LVR on materials prior to submitting them for adoption. To 
improve the guidelines the Department of Education collaborated with council 
members and publishers to clarify the LVR instructions and to properly 
interpret LVR reports. Everything was removed from the original guidelines 
except those items which were considered to be essential to providing evidence 
that LVR had indeed been carried out. 

The current 1985 Florida LVR Guidelines for publishers' reports are 
divided into four Sections: 




-12- 



13 



I. Product Background Inf armation 

In this section the publisher provides descriptive information, such 
as the publisher's name and address, title of textbook, author, 
copyright date and a signed statement certifying as to the accuracy of 
the content of the LVR report. This one page short description does 
little more than identify the materials that underwent LVR. It should 
be noted that publishers are required to submit an LVR report only on 
products which constitute the major tool for instruction. This 
typically applies to a textbook and not to student workbooks, teacher 
manuals, or other supplementary materials. However, it could also 
apply to computer software if the software is considered to be the 
"textbook 11 for the course. 

II. Intended Learner Outcomes 

The purpose of section two in the LVR Guidlelines is to determine if 
the developer used a systematic approach in development of the 
product. As mentioned earlier, LVR is part of a systems approach to 
improving instruction. This approach first starts with broad goals 
from which specific student-based measurable objectives are derived. 
Assessment items are then developed to make sure that the objectives 
are indeed measurable and also to serve as instruments to determine if 
the student has mastered the objective. Only after these two tasks 
are completed does the developer construct the instructional 
activities which comprise the lesson. If there is conguency between 
goals, objectives, and assessment items the developer is able to 
pinpoint problems in the instruction. Without these matching 
elements, improvement of instruction becomes a hit and miss 
proposition. 
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In this section the Publisher provides information indicating that 
the materials were constructed in a way that would allow for 
identification of problem areas for revision purposes. To help 
accomplish this, the publisher provides a sample of three goals, two 
objectives derived from each goal, and two test items which directly 
assess each objective. The expectation is that this information will 
provide a basis for revisions made in later sections of the report. 
III. Prepublication LVR 

This section is divided into four parts: 1. An overview of the LVR 
process used; 2. How the data was collected and used; 3. A listing 
of the instruments used; and 3. Selected samples of revisions. 

The overview is straightforward. The publisher describes on one 
page where, how, and by whom the process was carried out. There is no 
requirement that it be done in Florida. Since research shows on the 
one hand that information gathered from only one learner can make a 
difference and that information gathered from hundreds of learners is 
probably unusable for revision purposes, the guidelines suggest that 
the LVR be used with between 25 and 150 students of different academic 
abilities and socioeconomic conditions. One to six classrooms of 
students are sufficient. 

In collecting the data, publishers are required to use one or more 
of the following: observations, interviews, questionnaires/surveys, 
assessments. The oily stipulation is that students form the primary 
sources for the data. The publisher must report the type of person 
collecting the data, the characteristics of the students used, and 
most especially hew the data was used to make specific revisions. It 
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should be obvious that if a systems approach was used in developing 
the instruction the identification of trouble spots becomes more 
meaningful by directing the types of data to be collected and by 
showing the relationship between the data collected and the 
improvement decision. 

The publisher then lists the types of instruments used to collect 
the data and identifies those used directly with students. 

Finally f tne publisher attaches three samples of revisions made by 
including a copy of the revised and unrevised versions. These samples 
are expected to be those of high priority and not trivial corrections 
of typos or content inaccuracies. 
IV. Post Publication LVR 

LVR differs from the traditional formative evaluation process in that 
data from learners is collected during the entire lifetime of the 
product. In this section of the report, publishers are required to 
describe how data will be collected on the product while it is on the 
state adoption list which may be as long as six year. Publishers are 
not required to make any revisions during this interim period. 
Training and Inplenentation 

Not only have the guidelines been improved, but training for publishers 
in how to carry out the process and prepare the report has been provided. 
State council members are also trained in evaluating LVR reports. In this 
training, special attention is called to the fact that the LVR process and the 
validation process are quite distinct. Validation is a summative and not a 
formative process. Therefore, it usually requires a large number of students 
with strict control on how the materials are used. The confusion between 
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these two forms of evaluation has caused many problems in understanding and 
implementing the LVR process. 

Training of publishers consists of a one day workshop in which the LVR is 
the major component. Publishers are introduced to the concept of LVR and hew 
it became a requirement in Florida Statutes. The statutes are analyzed and 
the responsibilities of publishers are highlighted. Publishers then proceed 
through a series of hands-on activities in which they go through the 
guidelines component by component. Examples of good and bad reports are 
exhibited and discussed. The final activity involves the publisher 
critiquing one of their own reports associated with a previous submission. 
Feedback is provided as to how well they have performed. 

State council training in evaluating LVR reports forms part of a day and 
a half training program. One of the training modules explains the LVR 
process, the responsibilities of publishers and state council members, and an 
indepth study of the LVR guidelines. Council members are then trained through 
hands-on activities how to evaluate the components of the report. They are 
cautioned about being swayed with the use of large numbers of students as well 
as by the size of the report. Quality is more important than quantity. A 
check list is used to determine if all the components are included in an 
acceptable manner. The training concludes with the participants evaluating a 
simulated report. 

Publishers are required to provide a copy of the LVR report to each 
council member by August 1. This provides the members with several weeks, or 
in some cases months, to review the reports individually. They are encouraged 
to complete a check list for each report. When the councils assemble to make 
their final evaluation of the materials as a group, one of their first 
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activities is to arrive at a consensus on the acceptability of the LVR report. 
Before a set of materials is eligible for further consideration it must pass 
this scrutiny. 

Prior to 1983 it was almost unheard of to have a book declared ineligible 
due to an unacceptble LVR report. Since 1983 the number of rejections have 
risen. In 1984 there were 17% of the submissions rejected because of 
unacceptable LVR reports. That figure rose to 19% in 1985. Most of the 
rejections were due to incomplete items within the report or due to the fact 
that publishers had not done an LVR. Publishers of advanced or college texts 
have had special problems in that college divisions of publishing houses 
generally don't do LVR. Some have even been so bold as to state in their 
reports that they don't intend to have their materials go through the process. 
This attitude has resulted in several advanced texts being rejected. 
Implications 

Publishers are certainly not oblivious to the problems associated with 
fulfilling the requirements of the Florida LVR process. They have their own 
realworld concerns. Here are some of the issues raised as barriers to the use 
of the LVR process: 

1. Florida requires testing of the entire textbook rather than selected 
portions of it prior to publication. It is unrealistic and in fact impossible 
for publishers to try out an entire book in its final draft form for an entire 
school year. This is expensive, provides unrealiable data, and most of all 
does a disservice to those students being used as "guinea pigs." 

2. Students do not buy books. Teachers buy books. Therefore, it is 
necessary to get feedback only from teachers. Besides, the best test of a 
book is the marketplace. If it is a good book, it will sell. Ineffective 
books will be gradually weeded out. 
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3. Data gathered from students is highly unreliable. It is hardly 
useful for making informed judgments about revisions. 

4. Paperwork and more paperwork. All LVR seems to do is to raise the 
cost of the textbook. Who do you think will pay for the extra cost of LVR? 
Florida is doing a disservice to the rest of the schools in the country. 

5. Evidence is lacking that proves that LVR makes any difference in the 
effectiveness of the final product. Besides, no two so-called expects can 
agree as to an acceptable methodology. Exactly how many students are 
necessary? Are questionnaires alone sufficient? Do students have to be 
tested? Why? 

Individual state instructional materials council members vary in their 
acceptance of the LVR process. Some take their responsibilities very 
seriously. They scrutinize every LVR report and look carefully at the quality 
of what is in the report. Others are less concerned about LVR and interest 
themselves only with the quality of the books themselves. If the LVR report 
Stands in the way of whait they consider to be a "good" book from being 
adopted, they will disregard the LVR report. Still others, although having 
undergone training, misunderstand the process and either require too little or 
too much of publishers. 

The Department of Education is committed to the process. The statutes 
very clearly require that publishers use LVR in developing their materials. 
Although the statutes do not clearly indicate that an unacceptable report is 
grounds for rejection, the Department has always understood this to be the 
case and in recent years has seen to it that some submissions are rejected 
based on the LVR reports. 
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Learner verification and revision has been part of the Florida 
instructional materials state adoption process since 1974. Since that year 
Florida statutes require publishers to provide written proof that their 
materials were revised based on student feedback prior to publication. The 
statutes further require publishers to collect student data and to make 
revisions during the entire market life of the product. 

Up until 1983 Florida did not take LVR very seriously. Although LVR 
reports were required and given to state councils for their approval, no 
report was found unacceptable. This was largely due to the guidelines which 
were very generic. In 1983 the guidelines were made much more specific and it 
became easier to separate the acceptable from the non-acceptable reports. 
Council members were trained on how to review the reports. As a result, some 
reports were rejected in 1984 and again in 1985. 

The council reviews, however, are far from being consistent. There is 
much variablility within and between\counci Is. Some councils reject many more 
submissions than other councils. As a result, publishers are left confused 
and have grounds for further negative feelings about the process. 

In light of what is happening in Florida with an attempt to improve the 
quality of instructional materials through the LVR process the following 
recommendations are made: 

1. The guidelines should be continually reviewed to ensure that the 
integity of the LVR process is maintained within the bounds of the Florida 
Statutes. The guidelines should be reviewed for their clarity, consistency, 
and preciseness. 
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2. Publishers should cooperatively work with author-developers in 
planning and implementing the LVR process. In this way it is more likely that 
a systematic approach to developing materials will be used and that the LVR 
process will be used with integrity. 

3. To provide for consistency in applying the LVR guidelines to 
publishers' reports, a committee of specially trained council members, elected 
from each council, should meet to review all LVR reports prior to the 
evaluation of the materials. Besides providing for consistency and fairness 
this would streamline the evaluation process. 

Use of the LVR process does not guarantee quality textbooks, it may not 
even be in first place among all the methods suggested for improving books 
used in schools. But if the research literature is to be taken seriously use 
of the LVR process, when done properly, is way ahead of whatever is in second 
place. At least Florida thinks so. 
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